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Abstract: Intramural funding programs within both large and small research institutions are an 
essential mechanism to foster collaborative, novel, or preliminary research activity, and to further 
institutional research strategic goals. At most research institutions in the United States, thesefinding 
opportunities are managed by each funding agency or program, independently of each other. The 
lack of a shared administrative structure for these independent programs can lead to inefficiency 
and ineffectiveness of any one program. For example, a researcher must navigate several websites to 
determine which finding opportunities are available and then face multiple submission deadlines 
scattered across the entire academic year. The Resource Allocation Program (RAP), a shared 
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administrative structure at the University of California, San Francisco (UCSF), has centralized 
the announcement, submission and review process for opportunities offered by multiple intramural 
finding sources. Results of the shared structure include increased efficiency due to consolidation of 
internal review committees, increased visibility of funding opportunities, increased transparency and 
consistency of the selection process, and enhanced availability of valuable review feedback. 

Keywords: intramural finding, seedfunding, scientific review 

Introduction 

Intramural funding programs within both large and small research institutions are an essential 
mechanism to foster collaborative, novel, and preliminary research activity, as well as further 
institutional research strategic goals. Seed or pilot funding has been critical to individual research 
efforts, enabling researchers to establish experimental feasibility and generate preliminary 
data for more mature grant submission efforts. As Balaji and colleagues noted in their study, 
receipt of internal awards of even $20,000 can serve to jumpstart research projects and their 
subsequent higher-level funding (Balaji, Knisely, and Blazyk, 2007). Research institutions 
and their associated individual programs and departments can utilize such internal funding 
mechanisms to strategically support specific areas of research. The University of California, San 
Francisco (UCSF), like many large research institutions, has many internal funding programs 
sponsored by private foundations, large National Institutes of Flealth (NIH) funding-based 
centers, departments, and schools. 

At most research institutions, intramural funding opportunities are run independently of each 
other. In 2007, UCSF developed an intramural funding opportunity management program that 
accommodates multiple funding agencies/programs and grant mechanisms, simultaneously 
offering much added value to the campus-wide intramural funding enterprise. Flaving 
determined that good models for efficient intramural funding opportunity management did 
not exist at similar institutions, the UCSF Clinical and Translational Science Institute (CTSI) 
proposed, through funding from its NIFF CTSA grant (PI-Dr. Joseph McCune), to establish 
the Strategic Opportunities Support (SOS) Center that would support a seed grant funding 
program. The SOS Center developed an initial infrastructure for this seed funding program 
that allowed for efficient publicity, application and review of seed funding proposals. We soon 
realized this infrastructure could be utilized more broadly across the campus. 

In an effort to support the most promising novel research ideas and young investigators, 
as well as the work of established investigators and internationally recognized faculty, the 
SOS Center further improved its initial infrastructure for the distribution of seed funds. 
The new, improved infrastructure, based in part on the NIH’s Center for Scientific Review 
(CSR), was named the Resource Allocation Program (RAP) and was launched in the fall of 
2007. In 2011, RAP was placed under the centralized management of the Executive Vice 
Chancellor and Provost’s Office in order to realize the objective of supporting the entire 
campus and additional funding programs. The SOS Center continues its mission within 
CTSI and participates in the RAP consortium. 
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RAP is a campus-wide program responsible for coordinating intramural research funding 
opportunities. RAP serves as a consortium composed of numerous (currently 16; see Figure 1) 
UCSF funding agencies that may share overlapping goals while maintaining full autonomy over 
their funding mechanisms and awardees. (At UCSF, major funding programs are often referred 
to as “funding agencies”.) This cooperative venture between RAP and UCSF funding agencies 
awards nearly $5 million per year to UCSF faculty and distributed 145 awards during FY 2012- 
2013. RAP is designed to harmonize the award process by providing a standardized, centralized, 
and transparent process for the submission, review, and tracking of intramural research funding. 
This single-application process allows for a more efficient and cost-effective approach. 

Prior to the 2007 establishment of RAP, each UCSF funding agency ran independent 
competitions. A researcher had to navigate several websites to find out what funding 
opportunities were available and then face multiple deadlines, scattered across the entire 
academic year, in order to submit applications via primarily a “paper” or manual submission 
process. The review process was anything but uniform as each agency handled the review 
differently and reviewers were recruited on an as-needed basis. The reviewer pools utilized by 
each agency were fairly small and not able to offer broad skill sets. Conflicts of interest within 
each agency were a constant challenge during both the review and award process. 

Program Description 

RAP is a central institutional resource at UCSF, administered by the Executive Vice Chancellor 
and Provost’s Office and its component Research Development Office (RDO). Specific 
objectives of the program are to: 

• publicize intramural funding opportunities of member agencies and coordinate high- 
quality review of proposals by experts appropriate to the proposal topic; 

• coordinate funding of proposals among the funding agencies, maintain a database of prior 
proposals, and publicize successful applications on the Internet; 

• provide a forum for inter-agency communication regarding any aspect of the application 
process; and 

• carry out self-evaluation by developing metrics for success according to the RAP 
objectives. 

RAP’s benefits are well explained in its slogan, “One Application, Many Funding Opportunities, 
One Deadline.” Llorens and Kellough (2007) described the increased efficiency and performance 
of a “one-stop” recruitment process in their discussion of centralizing personnel administrative 
services. For similar reasons, RAP was transitioned centrally on campus in order to increase 
the overall efficiency of intramural funding processes; to increase the visibility, accessibility and 
ease of use for researchers; and to minimize the administrative redundancies of the application 
process among funding agencies. RAP provides a common application process and a common 
deadline for a wide range of intramural grant mechanisms. Funding opportunities are open to 
all UCSF appointees in all UCSF schools and affiliated sites. 
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Since its inception in 2007, RAP has grown from managing three intramural funding 
programs to the current 16 programs, whose funds come from a variety of intramural and 
extramural sources. RAP’s founding members comprised the UCSF School of Medicine 
Research Evaluation and Allocation Committee, the UCSF Gladstone Institute of Virology & 
Immunology Center for AIDS Research, and the CTSI SOS Center. UCSF’s Academic Senate 
and the Flelen Diller Family Comprehensive Cancer Center joined RAP at a later date. As 
additional funding agencies have joined RAP, the number of grant mechanisms has increased 
in number and diversity, as shown in Figure 1 below. 
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Figure 1. RAP Growth from Fall 2007 to Spring 2013. This figure includes only those 
agencies that use the full spectrum of RAP services: funding opportunity dissemination, 
submission and review management. Some intramural funding agencies choose to use only 
the dissemination and submission capabilities of RAP. 


Organizational Structure and Governance 

Bill Kirby, a long-time expert in research administration, described the need for programmatic 
commitment to quality, strong customer and stakeholder focus, and built-in continuous 
improvement processes in his Quality Management Model for successful research administration 
programs (Kirby, 1992). He went on to describe in a later paper the need to understand 
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research administration programs as “systems” or consortia of “interdependent components” 
or stakeholders (Kirby, 1996). The RAP founding leadership also recognized the value of group 
buy-in and stakeholder cooperation in order to establish a successful program structure. Thus, 
RAP has a multi-dimensional leadership structure that includes several entities, all playing a 
significant role in the program’s governance. 

The operational RAP leadership team includes the chair and vice-chair of the RAP Executive 
Committee, the RAP Program Manager, and the Research Development Office Director. As 
shown in Figure 2, the multi-dimensional leadership team is guided by two committees: an 
Executive Committee and a larger RAP Committee. 



Figure 2. RAP Organizational Structure 


The Executive Committee is composed of a chair, vice-chair, and program directors (all of 
whom are faculty members) whose programs have funded more than 10 percent of RAP 
awards in the previous two cycles. The Executive Committee provides guidance on program 
policy and operational issues, and meets on an as-needed basis. The committee’s chair and 
vice-chair are appointed by the UCSF Executive Vice Chancellor. The term of office for both 
is three years and is renewable. 

The RAP Committee, an essential aspect of the consortium-nature of the program, is composed 
of faculty directors and staff of the participating funding programs. Each funding program 
appoints one faculty member to the committee, as well as one staff member, usually a program 
manager, to improve up and down communication and increase effectiveness. Although faculty 
opinion is essential, we recognize that in order to achieve successful implementation, it is also 
important to involve staff members at the early stage of any discussion. Just as a scientist brings 
valuable perspective on medical and scientific issues, administrators can weigh in on the feasibility 
of programmatic matters such as budgeting, project timelines, and resource availability. 
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The RAP Committee regularly holds a monthly teleconference that allows all members to 
participate in continuous program improvement. RAP leadership requires consulting with the 
entire committee on significant changes regarding policy or major program improvements. 
This committee convenes in-person twice a year to make final funding decisions at the end of 
each cycle, and annually for a programmatic retreat. 

RAP Program Features 

Leslie Wimsatt and colleagues presented the highlights from a faculty workload survey 
conducted by the Faculty Standing Committee of the Federal Demonstration Partnership in 
Washington, D.C. (Wimsatt, Trice, and Langley, 2009). This 2009 survey highlighted the 
heavy administrative burden that is placed on our research faculty, with upwards of one-third 
of their time spent on pre- and post-award activities. Add to this the fact that grant funding 
success rates at NIFi, the largest funder of research grants in the US, has declined by nearly 30% 
since 1996, sitting at approximately 18% for R01 equivalent awards (Garrison & Drehman, 
2013), and we have a highly competitive funding environment and a limited amount of time 
available for researchers to spend on administrative tasks. For these reasons, among others, 
RAP has sought to optimize the ease-of-use to research applicants and maximize the potential 
funding success rate for any submitted proposal through a multi-faceted approach described 
in further detail below. 

Established Funding Cycles 

RAP offers two funding 
cycles per year: spring 
and fall. A typical cycle 
timeline is shown in Table 
1. Discrete, regular cycles 
are valuable to researchers 
and research administration 
staff who are involved with 
the process. Researchers are 
better able to plan their grant 
preparation responsibilities. 

In addition, this biennial 
cycle supports good planning 
and efficient time and 
resource management by 
administrators within both 
the RAP program and the 
funding agencies. 


Table 1. Typical Cycle Timeline 


Spring 2013 Cycle Timeline 

Date 

Task 

Mon January 7 

Provide Website/RFA changes 

Mon January 28 

Request for Applications 

Mon February 25 

Application Deadline 

Mon March 4 

List of applications is sent to RAP Executive Committee 

Fri March 8 

RAP EC assigns applications to Review Committees 

Thu March 14 

Applications are distributed to the chairs/support staff 

Thu March 21 

Review Committee chairs assign applications to reviewers 

April 29-May 7 

Review Committee meetings take place 

Mon May 13 

Scores are compiled and distributed to funding agencies 

May 17-May 24 

Funding agencies decide funding priorities 

May 28-June 5 

“Horse-Trading” Session to make final funding decisions 
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Comprehensive Marketing Strategy 

The launch of each cycle is supported by a multi-pronged marketing plan aimed at making 
the campus research community aware of available internal funding opportunities. RAP 
broadcasts the launch of each funding cycle broadly, highlighting changes such as the addition 
of new funding agencies or programs or the addition of new grant mechanisms. For each cycle, 
an email from campus research leadership on behalf of RAP is sent to faculty and researchers, a 
web article is published on the UCSF website, competition information is posted on the RAP 
website, and posters and flyers are distributed around campus. A sample poster publicizing the 
spring 2013 cycle is shown in Figure 3 below. 
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Figure 3. Sample Poster 
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RAP also supports a small number of “off-cycle” funding opportunities that are offered on an 
as-needed basis. In these cases, funding programs need to award their funds faster than the set 
deadlines of the main cycles would allow. Even in these cases, outside of the discrete cycles, 
the sponsoring programs are still able to leverage the RAP marketing strength as well as to 
utilize the same grant submission framework. The off-cycle grant opportunities, always offered 
as a single grant mechanism, are reviewed independently and do not use the RAP review 
committees. 

Straight-forward, Easy-to-use Submission Process 

RAP’s application, review, and award management process is shown in Figure 4 below. The 
Request for Applications generally goes out six weeks before the application deadline. The 
Request for Applications defines the application submission time window for the electronic 
submission system (the “start date” through the “end date”). On the start date, the electronic 
application form is active and the website is completely updated. 


Table 4. RAP Process Diagram 


Resource Allocation Program (RAP) 
Process Diagram 
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Every cycle is dynamic; new grant mechanisms can be offered and existing ones can be modified. 
New funding agencies can join RAP and existing agencies can decide which grants they want 
to continue to offer. The RAP Program Manager contacts participating programs and agencies 
ahead of the RFA announcement and asks them to review the website content relating to the 
grants they sponsor. Once this full assessment of all changes is done, the entire website content 
is updated using software engineering resources. The Program Manager is ultimately responsible 
for the deployment of a new and updated version of the website and electronic application 
before each new 


cycle is launched. In 
this way, any agency 
changes, e.g., types 
of grants offered or 
changes in eligibility 
requirements, 
are appropriately 
communicated. 

Effective Website 

The RAP website 
(http://rap.ucsf.edu), 
designed as a user- 
friendly, one-stop- 
shop for different 
intramural funding 
opportunities, is a 
significant resource 
for all users. 


University of California. San Francisco 


Resource Allocation Program 

A collaborative effort among UCSF funding agencies 



One Application 
Many Funding Opportunities 
One Deadline 

Two Cycles per Year 

Fall Cycle: September Deadline 
Spring Cycle: February/March Deadline 


How RAP Works 

The applicants choose the grant mechanism that best 
fits their proposals 

The funding agencies choose to fund the best proposals 
based on programmatic criteria 


Grant Mechanisms 

y Career Development 
> International & HIV-AIDS 
y Multidisciplinary 
>, Pilots 
y Technology 


DETERMINE ELIGIBILITY 
& COMPARE GRANTS 


Grant offerings, application rules, eligibility criteria, and other requirements are subject 
to change from cycle to cycle. Be sure to review the website after we launch the Call for 
Applications on August 12. 2013 to learn the updates and prepare your application 
accordingly. 


Fall 2013 Cycle 
Deadlines 


Call for Applications: 
August 12. 2013 


Application Deadline: 

September 23. 2013 
2:00 pm PST 


Award Notifications: 
January 2014 


Spring 2013 Cycle 
Deadlines 

Award 
Notifications: 
Mid-June 2013 


Feedback Notifications: 
End of June 2013 


UCSF-PUMCH 
Project Grants 

Deadlines 


Award Notifications: 
July-August 2013 


UPDATES 


Grants/Funding no Longer Offered 


• CC-CFAR Pilot Awards in HIV-Associated Malignancies 

• CTSI-Clinical Research Services (CRS) Early Career Investigators Pilot Award 

• Extra Funds for Eye Research - The Academic Senate will no longer fund eye 
research proposals at $50k (Ryan Endowment Fund no longer available) 


Completion of the Spring 2013 Cycle 

RAP reviewed 148 applications and made 72 awards for the Spring 2013 cycle; 49% of the 
grants reviewed were awarded Thanks to the combined efforts of multiple funding agencies, 
a total of $2,404,505 was distributed to UCSF researchers. For more details and to find out 
the names of the awardees go to 'Resources' on the RAP website 


Figure 5. Screenshot of RAP Website Home Page 


The Jotirnal of Research Administration, (45)1 





















90 Volpe, Kiser, Henry, Giacomini, Volberding, Waldman and Lowenstein 


The website is organized into the following sections: HOME, ABOUT, GRANTS, APPLY, 
RESOURCES, and CONTACT US. 

The HOME page, shown in Figure 4, highlights the program’s slogan, “One Application, 
Many Funding Opportunities, One Deadline,” and explains how RAP works. The page has 
information on RAP’s major grant areas and links to other pages with information on current 
opportunities, eligibility criteria, and award amounts. The HOME page also features updates 
on important policy changes, new offerings, and any news pertaining to RAP. There is a well- 
defined, color-coded section that highlights funding cycle deadlines. 

The ABOUT page briefly describes the program, mission, leadership team, and participating 
funding agencies. It also describes the RAP review process, and lists the standing review 
committees as well as all reviewers, without denoting their affiliation with a particular review 
committee. The GRANTS tab shows an overview of all grants offered, organized by major 
categories (Career Development, International, Multidisciplinary, Pilots, Technology). From 
this page, applicants can select each grant mechanism and find further specific information. 
The APPLY tab leads applicants to a step-by-step guide on how to submit a RAP application 
and includes an electronic submission form. 

The RESOURCES page provides information on past awardees, program statistics, descriptions 
of success stories, sample proposals, and letters of support. Finally, the CONTACT US page 
provides contact information for the RAP Program Manager and the administrators of each 
participating funding agency. 

One Simple Application Form 

Applicants submit proposals to RAP via an on-line application form that is the same for all 
grant mechanisms offered. Tie application is designed to capture all information needed for 
the review and award process as well as for future reporting. The current application form 
has evolved over the years with the feedback and general consensus from all the funders. The 
program tries to balance the need to capture as much information as possible with the benefit 
of keeping the application within a reasonable length. Changes to the application form are 
driven by the common utility among the funders. If a new piece of information is useful to 
the majority of funders and applicable to most grants offered, then the implementation is 
deemed valuable. 

Once all the required fields of the application form are completed, the applicant uploads the 
proposal as a single file in a PDF format. The proposals are written following mechanism- 
specific instructions, which are developed by the funders but harmonized to look similar and 
display the requirements in the same order so it is easy for the reviewers to identify missing 
or incomplete components. Any mechanism-specific proposal content is within the uploaded 
PDF file. After submission, applicants receive an email confirmation and have the opportunity 
to revise their proposals up until the submission deadline. RAP applications are submitted 
directly by the investigators who generally utilize their finance units or pre-award analysts to 
prepare the relatively simple budget. “Full-service,” formal institutional grant administrative 
support is not needed. 
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Past the deadline, the electronic application is disabled and the review process begins. All 
electronic submissions are stored in a database and later downloaded by the Program Manager 
and placed in a common data repository to be available for review. The database allows two 
levels of administrative access: a basic administrative level granted to all support staff, and a 
higher administrative level granted only to the Program Manager that allows deleting and 
adding applications on an as-needed basis. 

Comprehensive Expert Review 

The RAP review process has been modeled after the NIH CSR study section review process 
(NIH, 2012), which is used to review NIH grant applications for their scientific merit, 
providing unbiased, expert assessments. The RAP review process uses 10 specialized standing 
review committees and draws from a large campus-wide pool of internal faculty, matching 
reviewer expertise with each proposal to provide a fair, peer-reviewed process that will 
encourage and stimulate research. This approach is possible at UCSF because it is a large 
institution with a broad spectrum of scientific areas represented. In a different setting the 
process might require the recruitment of external reviewers. It has been challenging to balance 
the benefits of utilizing senior faculty, who are more experienced, with junior faculty, who 
are more available and willing to serve. We try to achieve a good mix of senior and junior 
faculty. This balance also provides the perfect setting for mentoring opportunities. The review 
committee chairs are always senior faculty and expert reviewers. Finally, it is our goal during 
the reviewer recruitment process to have a good representation from all the schools: Medicine, 
Pharmacy, Dentistry, and Nursing. 

Most applications submitted through RAP are assigned to a review committee organized by 
subject area. Currently, RAP manages 10 review committees. These committees are standing 
committees, each led by a chair and composed of UCSF faculty members. Each review 
committee reassesses subject areas, chairs, and membership on a yearly basis. Faculty members 
are nominated based on suggestions from the funding agencies, the chairs, and members of 
review committees. Nominees are also drawn from past awardees and other faculty who have 
volunteered for these committees. 

Proposals are reviewed by at least two individuals with expertise in a closely related field and 
discussed by the entire topical review committee. Proposals are scored according to the review 
criteria delineated by the funding programs in the descriptions of each grant mechanism. 
Critiques are returned to the proposal Principal Investigator as feedback. Funding agencies 
choose to fund proposals based on the review comments, scores, and respective alignment with 
their programmatic goals. 

The RAP review committees were formed according to the needs defined by the funding 
mechanisms offered through RAP and may be modified over time based on the proposals 
submitted. The program’s goal is to establish a large pool of experts in various research areas 
and have all committee members be potential reviewers. However, depending on the number 
of proposals received and the reviewers’ areas of expertise and availability, each reviewer will not 
necessarily review every cycle. Before RAP, each program could only use faculty reviewers who 
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were part of specific networks. The pools were small and often the match between application 
and reviewer expertise was far from ideal. 

The RAP review process is perceived as fair by faculty leadership, reviewers, and applicants. 
Great attention goes to avoid conflict of interest during the review process, and chairs make a 
sincere effort to match each application with the appropriate expert reviewers. There is a strong 
commitment to provide a good review as each reviewer could be an applicant in the next cycle, 
and therefore everyone desires a fair process. 

RAP Proposal Review Process 

Review Committees 

The RAP Executive Committee receives the full list of applications and can access all proposals 
via Dropbox®. The Program Manager makes the first attempt at assigning applications to the 
appropriate review committee. All resubmissions are reassigned to the same committee and 
to the same reviewers who did the review previously. The Executive Committee confirms 
assignment of each application to the appropriate review committee by topic. During the 
online submission, applicants can select the review committee, but the final decision regarding 
committee assignments is ultimately up to the Executive Committee and later confirmed by 
review committee chairs. Applicants are also given the opportunity to indicate one potential 
faculty member who should not review their proposal; chairs do their best to accommodate 
their request. 

The chair of a RAP review committee plays a key role in the functioning and success of the 
review process. The chair and administrative support staff are responsible for managing the 
review process in compliance with RAP policy. The chair and support staff work in partnership 
to help facilitate a review process of the highest quality and fairness. 

Review committee chairs receive and review the applications assigned to their committees 
and either agree with the assignments or suggest alternative review committees better suited 
in expertise. The review committee chair assigns two reviewers to each proposal, each equal in 
responsibility. The chair tries to avoid even the appearance of a conflict of interest and takes 
into consideration both the proposal PI(s) and the co-investigators. For RAP, being in the same 
department does not necessarily constitute conflict of interest. Faculty members with the same 
skill set most likely work in the same unit and are the experts needed for a good review. The 
general guideline for conflict of interest is that reviewers are in conflict if their research will 
benefit directly if the proposal is awarded, if they are in direct competition with the applicant 
in the research arena, or if they have a personal relationship with the applicant that may lead to 
bias. The larger the size of a review committee, the easier it is to manage the conflicts. 

Once the reviewers’ assignments are complete, support staff members contact each reviewer 
and provide materials necessary to complete the review of the grants assigned. This includes: 

• a full list of all proposals assigned to that committee; 
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• customized review forms saved with a specific naming convention that makes it easy to 
retrieve reviews later. The review forms display the 1-9 scoring system used by RAP and 
scoring rubrics. Additionally, the forms have embedded any grant mechanism-specific 
questions that the reviewers need to address in order to provide a comprehensive review in 
alignment with research standards and funders’ goals; 

• a request to reply immediately if there is a conflict of interest so the proposal can be 
reassigned; and 

• standard operating procedures (SOPs) developed to orient and guide chairs, reviewers, 
and support staff on the review process and on how to conduct the review committee 
meetings. The RAP review process SOPs are in alignment with the NIH review criteria 
and describe in great detail pre-meeting, meeting and post-meeting responsibilities to 
ensure consistency across the review committees. 

Reviewers can access all applications assigned to their committee and previous reviews for the 
re-submissions. Reviewers are given approximately four weeks to review their set of proposals 
and assign preliminary overall scores based on the NIH scoring system. RAP does not ask 
reviewers to assign a score to specific review sub-categories as is done by the NIH CSR study 
sections. 

Review Committee Meetings 

Prior to the review committee sessions, the chair and support staff discuss plans and materials 
needed for the review session. The chair reads the reviews in advance of the meeting to facilitate 
the discussion, especially for those applications with a wide divergence in preliminary scores. 
An effort is made to have as many participants as possible attend in person with the option 
of teleconferencing offered as needed. The chair is responsible for ensuring each review is fair, 
equitable, and free of bias. 

All proposals receiving a score of 5 or greater from both reviewers are triaged and not discussed, 
unless there are members of the review committee who would prefer to discuss them. The 
assigned reviewers provide their initial level of enthusiasm and concise reviews of the application 
with emphasis on its impact, strengths, and weaknesses. The proposal is then discussed by all 
committee members. Each proposal is given a final score by all non-conflicted reviewers in 
attendance (in person or by phone), including the chair. (These scores will be averaged to 
generate a single final score.) At the end of the meeting, scoring sheets are collected, and issues 
are either clarified or noted. 

Post-Review Session Responsibilities 

After the meeting, final scores are tabulated and results sent to the Program Manager. Reviewers’ 
attendance and workload are tracked for all committees every cycle. Reviews for all the proposals 
in the competition are collected and reviewers’ names and initial scores are redacted. 

The Program Manager distributes the final results in a spreadsheet to the RAP committee 
(funding agencies). The sheet includes all applicants’ names and proposals titles, the average 
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and standard deviation of the review committee final scores, and special notes. The sheet is 
organized by scores (from best to worse) and by grant category. The spreadsheet also contains 
charts with the distribution of scores among each committee and overall. Redacted reviews are 
made available to the funding agencies to facilitate their funding decision process. 

Funding Agency Meetings 

Using this list of candidate proposals and the associated final scores, each funding program 
plans its own internal meeting to decide funding priorities. During this meeting the funder 
discusses which proposals they would like to see funded, regardless of their ability to fund, 
and which proposals they would specifically prefer to fund. Funding agencies take reviews 
and scores into consideration along with how well a proposal fits their mission when deciding 
which proposals to fund. 

The Cooperative “Horse- Trading” Session — Final Funding Decisions 

One of the novel and valuable steps in the RAP process is the final funding session. All faculty 
directors and managers of the participating funding agencies, along with RAP staff, convene 
in person to make final funding decisions. The RAP Executive Committee chair moderates 
the session, calling every proposal in ascending order of score (best to worst) and asking who, 
among the funders, is interested in funding that application. The process is often referred to 
as “horse-trading” since it frequently involves trading one proposal for another to ensure all 
agencies meet their funding goals. 

Our experience has shown that this meeting is remarkably successful in accommodating 
everyone’s needs and allowing for a higher number of proposals to be funded. All funders come 
into the meeting with the goal of allocating their funds strategically and supporting as much 
good science as possible. This step in the RAP process significantly improves the likelihood of 
funding a strong proposal. Proposals submitted to one program may be picked up for funding 
by another funding agency because the topic of the proposal is relevant to that agency as well. 
Sometime proposals are co-funded by two different funding agencies. 

After the meeting the results are circulated one additional time and agencies have a chance 
to review the funding decisions and make any necessary adjustments. If there are no issues, 
funding decisions become final and the funding agencies can start notifying their awardees 
while the RAP Program Manager sends out declination letters. 

Applicant Feedback 

At the end of each cycle, all applicants are notified of the outcomes and receive written reviews. 
The reviews are redacted so as not to reveal scores, reviewers’ names or confidential comments. 
As has been observed before (Balaji et al., 2007), regardless of the funding outcome, this 
feedback is extremely valuable to applicants, especially for either their resubmissions to 
intramural funding opportunities or for subsequent extramural grant development. At the 
final step, the names of the awardees are posted on the RAP website along with the funding 
statistics for the cycle. 
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Costing Models 

All programs on campus are eligible and welcome to use RAP to coordinate their funding 
opportunities via one of the two costing models: 

Model 1: A full-service option that includes dissemination, submission, and review of proposals. 
Each program is charged based on the percentage of dollar amount funded in the prior academic 
year, i.e., their share of total cost is in direct proportion to the amount they awarded. 

Model 2: A partial service option, where a funding agency uses RAP to publicize funding 
opportunities and administer on-line submissions. Costs are assessed on the basis of proposal 
submissions, i.e., there is a fixed charge per proposal submitted. 

RAP has also served some programs by only advertising their funding opportunities at no cost. 

A quick glance at the RAP budget will show that it is fairly lean and simple. It is composed 
of personnel expenses covering the Program Manager’s salary and, in small percentages, the 
salaries of the other leadership members (RDO Director, RAP Chair and Vice-Chair). The 
budget also includes items such as honoraria for review committee chairs, computer support, 
marketing materials, and supplies. 

The funding agencies that have used RAP recognize the added value and appreciate the services 
provided. Institutional leadership (Executive Vice Chancellor and deans of the four schools) 
has also recognized the value and benefits to the campus community and, in fact, partially 
subsidizes the program. The RAP model requires central support to succeed so that the costs 
are shared between the funding programs and the institution. Although RAP provides a huge 
value-add compared to what each single program alone could do, it can be too much of a 
financial burden, especially for the smaller programs, to pay for premium grant competition 
management services relative to other competing needs within their specific programs. 

Implementation Challenges 

Although the program founders had a clear vision and strongly believed that RAP would 
improve the management of intramural funds on campus, like in many process changes, there 
were some concerns. The funding programs and agencies were concerned that joining RAP 
would diminish their unique identity and perhaps even create confusion about their specific 
programmatic goals. The concern over programmatic identity was soon overcome by a simple 
marketing strategy. Agencies were represented on the RAP website and announcement 
materials with their names and logos, such that applicants easily identified the funding 
programs. The website continues to highlight the supporting funding program(s) for each grant 
mechanism offered. Some grant mechanisms are only supported by one funding program; 
others, e.g., the Pilot for Junior Investigators, are supported by several programs depending 
on the proposed project topic area and funder strategic goals. The Research Evaluation and 
Allocation Committee grants, for example, only support researchers affiliated with the School 
of Medicine and the Helen Diller Family Comprehensive Cancer Center requires faculty 
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applicants to be Cancer Center affiliates, while programs such as the Academic Senate have a 
broader funding target and fewer restrictions. 

Another concern expressed by the early RAP adopters was that they might not maintain full 
autonomy over their funding mechanisms and awardee selection. This concern disappeared 
when programs realized they had full control over their research funding announcement 
content and grant descriptions. They were (and are) still able to determine independently 
key components of the grant mechanism, eligibility, review criteria, and award amount. They 
also provide grant-specific instructions that capture additional relevant applicant information. 
Occasionally RAP encourages funders to follow similar parameters when providing grant 
descriptions and instructions if that can benefit the applicants and the overall application 
process. In those cases, the applicant ease-of-use and positive agency outcome has significantly 
outweighed any small loss of funder-relevant application information. RAP has encouraged 
all the participating agencies/programs to focus on the big picture and think beyond the 
confines of their own programmatic goals to explore together what can be done for the sake of 
supporting good science and collectively helping out the investigators at our institution. 

Administratively, it can be challenging to change “the old way of doing things.” It is hard to 
change well-established methods and replace them with new practices. To that point, good 
communication between faculty directors and administrators has been a crucial and positive 
step forward itself. Successful process change requires an understanding of the change rationale, 
and the conviction that trying something new is worth it. During the initial implementation 
phase of RAP, we were fortunate that several funding program staff members involved were 
new, making things easier as no one was too strongly attached to old procedures. Inspired 
by the desire to implement innovation, the new and old funding program staff embraced 
the change to a centralized management structure. In addition, RAP maintains a strong 
“consortium nature,” with scheduled collaborative discussion that includes participants at all 
levels of the process (see discussion below). 

Finally, coordinating staff members who support the review committees but do not directly 
report to the RAP Program Manager or to the RAP central leadership requires tact and 
consideration. Central guidance is indisputably needed to keep the process consistent, but to 
make things work, it is important to establish a positive model of partnership and “managing 
without authority.” Staff members involved need to buy-in and share the RAP programmatic 
goals, recognizing the value of each single contribution. Open communication is crucial so that 
all parties involved feel comfortable expressing their opinions, whether they be reservations 
or approvals. RAP management has worked hard through the organizational operations to 
empower each individual agency to speak its mind while encouraging everyone to keep an 
eye on each others needs as well. By fostering respectful communication, RAP management 
strengthens the shared belief that a strong working relationship among the RAP participants is 
necessary to achieve institutional intramural funding goals. 
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Results 

This model for managing multiple intramural funding mechanisms has been very successful, 
owing in large part to the founders’ clear vision regarding the program’s purpose and goals. The 
program is infused with a sense of shared responsibility and accountability to the greater good 
of research at UCSF. Stakeholders still maintain a strong sense of ownership and commitment, 
and are proud of this fruitful collaboration uniquely characterized by strong involvement of 
funders, faculty, administrative staff, and central campus leadership. 

All stakeholders benefit from a more formalized, standardized application process. RAP 
provides an ideal infrastructure for sharing best practices, enables staff networking, and 
realizes administrative efficiencies and economies of scale. RAP is able to reduce overlapping 
administrative functions and, at the same time, provide additional administrative functionality. 
Importantly, RAP maintains funding agencies’ identities, and provides increased visibility of 
smaller funding programs. 

A significant benefit of the RAP process is that researchers have an increased likelihood of 
receiving research funding (compared to extramural funding), while the funding agencies often 
see increased funding of proposals of their strategic interest. This synergy is attained through 
the horse-trading session where innovative proposals submitted to one agency mechanism can 
be co-funded or funded wholly by another agency. In this way, RAP enables the funding 
of strong proposals within an agency’s interest but beyond their usual budget limits, and it 
enables an increase in the necessary seed support for campus research programs. 

Not only are intramural funding programs like RAP inherently valuable for enabling the 
generation of preliminary data needed for subsequent grant proposals, but they also enable the 
opportunity for applicant mentoring and proposal quality improvement (Balaji et al., 2007). 
RAP is no exception in this regard because it fosters mentoring within the review committee 
setting and enables proposal quality improvements via quality written feedback from the 
expert review process. Additionally, channels are created for networking of applicants with 
past awardees and review committee members. 

Because the RAP review committees draw reviewers from the entire campus and hold single 
review committee meetings, they consistently offer broader scientific coverage, save reviewers’ 
time and reduce their workload, and lower the potential for conflict of interest. This process 
also creates opportunities for mentorship of junior faculty reviewers. The expert review 
committee structure additionally enhances selection transparency and agency accountability 
for their funding decisions. 

The RAP process provides tracking of all submissions, enables generation of statistical data 
on submissions, and stimulates agencies to revisit their goals/rules. RAP is working toward 
helping agencies in the evaluation and administration of awardees, e.g., tracking outcomes to 
evaluate the return on investment, facilitating progress reporting, and monitoring expenditure 
of funds. 
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Logistically, RAP offers other benefits to both researchers and stakeholders, including the fact 
that grant opportunities are easy to find as they are all in one web location, one application 
form fits all grant mechanisms, and there are two recurring cycles per year with a single deadline 
per cycle. 

The program has grown steadily in support of different funding agencies since its launch in 
2007. In FY2012-2013, RAP facilitated the award of $4.6 million to a broad spectrum of 
UCSF researchers. All four schools (Schools of Medicine, Pharmacy, Nursing and Dentistry) 
and the Graduate Division are routinely represented among the awardees, and a full range of 
biomedical research is supported through RAP funding. Due to its effectiveness and strong 
marketing strategies, RAP is now one of the most highly recognized programs on campus. 
Moreover, another program at UCSF was modeled after RAP, the Resource Allocation Program 
for Trainees (RAPtr), which was created to facilitate funding opportunities for students and 
trainees involved in research or innovation projects. 

Figure 6 shows the growth in overall grant submissions regardless of the type of review (All 
Submissions) and grant submissions that utilized all aspects of the RAP submission and review 
process (RAP-reviewed Submissions). Of those grants that did go through the full RAP review 
process, the graph indicates the number of grants awarded per cycle and the total dollar amount 
awarded for that cycle. 



RAP Cycles 

Figure 6. RAP Process Diagram 


The centralized dissemination of funding opportunities not only connects researchers to 
potential funding in an organized, consistent, and simple way, but also provides university¬ 
wide visibility to the participating funding agencies. From the funding program perspective, 
this is certainly one of the biggest strengths of RAP. Table 2 shows the impact of RAP process 
management for one of the funding programs - the UCSF Gladstone Institute of Virology & 
Immunology Center for AIDS Research (CFAR). The number of applications received for 
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Pre-RAP 

Management 

Post-RAP 

Management 

Year 

2006 

2007 

2008 

2009 

2010 

2011 

Number of 
Applications 

16 

14 

25 

37 

47 

46 


Table 2. CFAR Grant Application Number before and after RAP Implementation 

funding consideration nearly doubled after implementing RAP and its associated “marketing 
strategies.” Anecdotally, this program also saw awards become more broadly distributed across 
the four schools on campus post-RAP implementation (data not shown). 

RAP has also provided a structure through which funding agency programmatic data can 
be captured; prior to RAP most programs did not have the resources to track program 
performance metrics. Through RAP, funders now have access to more comprehensive statistics 
and have developed an awareness of the importance of accurate metrics. We continue to seek 
improvements in performance assessment. 


UCSF Clinical and Translational Research Institute 
Strategic Opportunities Support (SOS) 
Return on Investment by Fund Year 

$30,000,000 


$25,000,000 

$ 20 , 000,000 
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Figure 7. Return on Investment from the CTSI-SOS Program, one of the RAP funding 
agencies. The consistent and significant return on initial pilot funding program investment is 
largely due to the award of larger extramurally-funded grants. 
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RAP has facilitated the success of many on-campus funding programs such as CFAR and the 
CTSI-SOS Center. While it is hard to conclude what would have happened had an agency 
like CTSI-SOS not used RAP, it is evident that the investigators selected through the RAP 
process were successful at leveraging the results of pilot funding for successful bids to obtain 
larger extramural grant funding. Figure 7 shows the return on financial investment for the 
CTSI-SOS program, which has been able to parlay awards from RAP-funded grants into larger 
research awards worth six- to sixteen times more. Similarly, CFAR has reported that with an 
investment of $3,032,038 between 2007 and 2011, they have seen an increase in the numbers 
of K, R, P and U federal awards involving HIV/AIDS and totaling more than $135 million 
(see Figure 8). 


CFAR 

Pilot Total 



Awards 

K Awards 

R Awards 

P Awards 

U Awards 

Awarded 

Eradtcatioalatency 

120,000 

835,155 

1,731,403 

5.639,865 

5,142,040 

13,468,463 

Biomedical Prevention 

15,000 

243,308 

5,300,711 

0 

7,724,303 

13,283,322 

HIV and Women 

465,000 

1.161.036 

6,370,934 

2.858,691 

16,274,838 

27,130,499 

HIV and Cancer 

827,353 

0 

4,825,334 

533,743 

7,703,786 

13,890,216 

HIV and Aging 

190,000 

2,454,242 

16,178,815 

0 

3,735,540 

22,558,597 

International 

1.414,685 

4,629,347 

20,335.858 

7.485,516 

11,034,188 

44,899,594 

Sub-Total 

S3.032.038 

$9,323,088 

$54,743,055 

$16,517,815 

$51,614,695 

$135,230,691 


Figure 8. Productivity from UCSF - Gladstone Institute of Virology & Immunology Center 
for AIDS Research (CFAR) Pilot Grants 


The success of these programs can also be measured in the number of scholarly publications 
that have resulted from proposals funded through RAP. CFAR, for example, reports that 
between 2007 and 2011, 87 of these award recipients and mentees have contributed to 469 
peer-reviewed publications related to their studies (Figure 8). Prior to RAP, the UCSF funding 
programs did not track much information, but have now matured through RAP-fostered 
discussions and have an understanding of the importance of implementing systems that allow 
for tracking and reporting. This illustrates another benefit of centralizing the management of 
intramural funding: all participants gain insight leading to improvements in operations not 
directly related to fund competition management itself. 

Conclusion 

The RAP intramural funding process offers a variety of benefits to both researchers and 
stakeholders. The existence of a centralized, coordinated program allows researchers to 
easily find funding opportunities and plan their grant-writing responsibilities, since the two 
cycle deadlines per year are recurring and predefined. The RAP grant format itself is also 
simple and consistent across multiple mechanisms. This ease-of-use helps investigators access 
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research funding at critical junctures in their research career, and is especially important 
because the funding rate for applications is usually close to 50%, much higher than most 
other funding opportunities. 

Researchers can also benefit from the networking channels created through the program with 
past awardees as well as among review committee members. The RAP review process offers an 
ideal setting for mentoring of junior faculty reviewers by more experienced colleagues. The 
quality of the reviews produced through RAP is optimized because of the substantial effort 
done by the review committees to match each application with the most qualified expert in the 
field across campus. The feedback provided to applicants helps with both RAP resubmissions 
and grants submitted to other internal or external mechanisms. 

Applicants are not the only beneficiaries of the system, however. The consortium nature of 
the RAP process enhances transparency and accountability of all parties involved, challenging 
all to operate at their best, to have an open view on issues, and to rethink and continuously 
improve their processes. Stakeholders, such as the funding agency staff and faculty volunteers, 
also gain from RAP and they recognize the added value: RAP provides a more formalized 
and standardized process that leads to more consistent reviews and funding decisions. All 
participating agencies are now able to track all submissions (funded/non-funded) and can 
generate statistical data for reporting purposes. This consortium process constantly stimulates 
agencies to revisit and improve their goals and rules, and provides an excellent infrastructure in 
which they can share best practices and find opportunities for networking among themselves. 

The next big step in the RAP program development is implementing a robust electronic 
solution that will not only enable efficient submission, review, award and tracking of proposals, 
but robust management and reporting as well. We are now ready to streamline the review 
process and decrease the administrative burden through a cost-effect electronic system. The 
award process, handled by each agency, can also be improved. If all data, from submission to 
award, is kept in the same system, RAP will be able to facilitate the development of reporting 
best practices for all participating funders. Identifying and developing such a tool and 
implementing it successfully are our next big challenge. 

One of the programs biggest strengths has been the stakeholders’ clear vision and cooperative 
behavior. It is our hope that RAP will be seen as a model and will inspire other institutions, big 
and small, to consider implementing a centralized intramural funding management program. 
Although the RAP process cannot be described as simple—it is actually more complex than the 
singular processes used in the past by the separate agencies—participants agree there is essentially 
no redundancy of work and every single effort is driven by a strong purpose. The deans and 
the Executive Vice Chancellor have recognized the value-add of a centralized management 
mechanism and are fully supportive of the program, recognizing that it strives to guarantee 
transparency, fairness and efficiency while maximizing the investigator potential for funding. 
Together, the RAP staff and associated consortium of funders and faculty do more and try to 
do it better - truly a great example of on-campus synergy and teamwork. 
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